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Executive Summary 


Race to the Top overview 


On February 17, 2009, President Obama signed into law the 
American Recovery and Reinvestment Act of 2009 (ARRA), 
historic legislation designed to stimulate the economy, support 
job creation, and invest in critical sectors, including education. 
ARRA provided $4.35 billion for the Race to the Top fund, of 
which approximately $4 billion was used to fund comprehensive 
statewide reform grants under the Race to the Top program." In 
2010, the U.S. Department of Education (Department) awarded 
Race to the Top Phase 1 and Phase 2 grants to 11 States 

and the District of Columbia. The Race to the Top program is 

a competitive four-year grant program designed to encourage 
and reward States that are creating the conditions for education 
innovation and reform; achieving significant improvement in 
student outcomes, including making substantial gains in student 
achievement, closing achievement gaps, and improving high 
school graduation rates; and ensuring students are prepared 
for success in college and careers. Since the Race to the 

Top Phase 1 and 2 competitions, the Department has made 
additional grants under the Race to the Top Phase 3, Race 

to the Top — Early Learning Challenge,” and Race to the Top — 
District? competitions. 


The Race to the Top program is built on the framework of 
comprehensive reform in four education reform areas: 


« Adopting rigorous standards and assessments that prepare 
students for success in college and the workplace; 


« Building data systems that measure student success and 
inform teachers and principals how they can improve their 
practices; 


« Recruiting, developing, retaining, and rewarding effective 
teachers and principals; and 


¢ Turning around the lowest-performing schools. 


Since education is a complex system, sustained and lasting 
instructional improvement in classrooms, schools, local 
educational agencies (LEAs), and States will not be achieved 
through piecemeal change. Race to the Top builds on the 

local contexts of States and LEAs participating in the State’s 
Race to the Top plan (participating LEAs) in the design and 
implementation of the most effective and innovative approaches 
that meet the needs of their educators, students, and families. 


' The remaining funds were awarded under the Race to the Top Assessment 
program. More information about the Race to the Top Assessment program is 
available at www.ed.gov/programs/racetothetop-assessment. 


More information on the Race to the Top — Early Learning Challenge can be 
found at http://www2.ed.gov/programs/racetothetop-earlylearningchallenge/ 
index.html. 


More information on Race to the Top — District can be found at http://www2. 
ed.gov/programs/racetothetop-district/index.html. 


Participating local educational agencies (LEAs) are those LEAs that choose to 
work with the State to implement all or significant portions of the State’s Race 
to the Top plan, as specified in each LEA’s Memorandum of Understanding 
with the State. Each participating LEA that receives funding under Title |, Part 
Awill receive a share of the 50 percent of a State’s grant award that the State 
must subgrant to LEAs, based on the LEA’s relative share of Title |, Part A 
allocations in the most recent year, in accordance with section 14006(c) of the 
American Recovery and Reinvestment Act (ARRA). 
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Race to the Top program review 


As part of the Department’s commitment to supporting States as they 
implement ambitious reform agendas, the Department established 

the Implementation and Support Unit (ISU) in the Office of the 
Deputy Secretary to administer, among others, the Race to the Top 
program. ‘The goal of the ISU was to provide assistance to States 

as they implement unprecedented and comprehensive reforms to 
improve student outcomes. Consistent with this goal, the Department 
has developed a Race to the Top program review process that not 

only addresses the Department’s responsibilities for fiscal and 
programmatic oversight, but is also designed to identify areas in which 
Race to the Top grantees need assistance and support to meet their 
goals. Specifically, the ISU worked with Race to the Top grantees 

to differentiate support based on individual State needs, and helped 
States work with each other and with experts to achieve and sustain 
educational reforms that improve student outcomes. In partnership 
with the ISU, the Reform Support Network (RSN) offers collective 
and individualized technical assistance and resources to Race to 

the Top grantees. The RSN’s purpose is to support Race to the Top 
grantees as they implement reforms in education policy and practice, 
learn from each other, and build their capacity to sustain these 
reforms.’ At the end of Year 4, the Department created the Office of 
State Support to continue to provide support to States across programs 
as they implement comprehensive reforms. The Office of State 
Support will administer programs previously administered by the ISU. 


Grantees are accountable for the implementation of their approved 
Race to the Top plans, and the information and data gathered 
throughout the program review process help to inform the 
Department’s management and support of the Race to the Top 
grantees, as well as provide appropriate and timely updates to the 
public on their progress. In the event that adjustments are required 
to an approved plan, the grantee must submit a formal amendment 
request to the Department for consideration. States may submit for 
Department approval amendment requests to a plan and budget, 
provided such changes do not significantly affect the scope or 
objectives of the approved plans. In the event that the Department 
determines that a grantee is not meeting its goals, activities, timelines, 
budget, or annual targets, or is not fulfilling other applicable 
requirements, the Department will take appropriate enforcement 
action(s), consistent with 34 CFR section 80.43 in the Education 
Department General Administrative Regulations (EDGAR).° 


5 More information can be found at http://www2.ed.gov/about/inits/ed/ 
implementation-support-unit/tech-assist/index.html. 


5 More information about the Implementation and Support Unit’s (ISU’s) program 
review process, State Annual Performance Report (APR) data, and State Scopes 
of Work can be found at http://www2.ed.gov/programs/racetothetop/index.html. 
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State-specific summary report 


The Department uses the information gathered during the review 
process (¢.g., through monthly calls, onsite reviews, and Annual 
Performance Reports (APRs)) to draft State-specific summary reports. 
‘The State-specific summary report serves as an assessment of a State’s 
annual Race to the Top implementation. The Year 4 report for 

Phase 2 grantees highlights successes and accomplishments, identifies 
challenges, and provides lessons learned from implementation from 
approximately September 2013 through September 2014. Given that 
Delaware and Tennessee’s initial four-year grant periods ended in June 
and July 2014, respectively, for Phase 1 grantees, the Year 4 report 
includes the beginning of the no-cost extension year (Year 5). 


The State’s education reform agenda 


Since the passage of the 1993 Education Reform Act, Massachusetts 
has focused on accelerating student achievement across the State. In 
2011, Massachusetts’ fourth and eighth graders led the Nation in 
reading and mathematics performance on the National Assessment 
of Educational Progress (NAEP). Despite having high overall levels 
of student achievement, Massachusetts recognizes that achievement 
gaps persist and not every student in the State receives a world-class 
education. Through Race to the Top, Massachusetts implemented 

a comprehensive reform plan to provide students with the tools, 
supports, and technology to help ensure that every student is prepared 
for success in college and career in the twenty-first century. 


Massachusetts’ Race to the Top grant of $250,000,000 supports 

the State’s commitment to transform teaching and learning in 
classrooms across the State, improve student performance, and close 
achievement gaps. In keeping with the terms of the Race to the Top 
grant, Massachusetts is using half of its grant funds to drive State-level 
work, and distributing the other half of its award to support work in 
participating LEAs that is aligned with the State’s goals. 


‘The State’s Race to the Top grant is focused around four primary 
objectives: 


(1) attract, develop, and retain an effective, academically capable, diverse, 
and culturally proficient educator workforce to ensure that every 
student is taught by a great teacher, and every school and district is 
led by a great leader; 


(2) provide curricular and instructional resources to give every educator 
the tools necessary to promote and support student achievement; 


(3) concentrate great instruction and additional supports for educators, 
students, and families in the lowest-performing schools and their 
districts to create the conditions needed for improved student 
achievement; and 


(4) increase dramatically the number of students who graduate from 
high school ready for college and career. 


State Years 1 through 3 summary 


In the first two years of the grant period, the Massachusetts 
Department of Elementary and Secondary Education (ESE) 
established systems to oversee and manage Race to the Top reforms 
and projects. The Delivery Team within the Office of Planning and 
Research (OPR) fully implemented the Delivery approach, which 
emphasizes the use of real-time data, focused analysis and reports, and 
strong leadership involvement to drive implementation. Also, early 

in the grant period the State collaborated with key stakeholders in 
developing each component of the State’s Race to the Top plan, which 
increased buy-in among educators. 


After a two-year transition period, Massachusetts educators began 
implementing the Massachusetts Curriculum Frameworks in 
English language arts (ELA) and mathematics, which incorporate 
the Common Core State Standards (CCSS), in school year (SY) 
2012-2013. To support local implementation in Years 2 and 3, ESE 
provided a variety of resources through its website and the Edwin 
Teaching and Learning system including teacher-developed model 
curriculum units, State-led regional trainings, and instructional 
resources like curriculum maps. After a small pilot in Year 2, the 
State released the Edwin Teaching and Learning system in Year 3, 
which makes these resources available electronically to all educators. 
In addition, the State made progress in other college- and career- 
readiness initiatives, including opening six science, technology, 
engineering, and mathematics (STEM) Early College High Schools 
and supporting 47 Innovation Schools. 


‘The State’s progress with data systems was varied during Years 1 
through 3. After procurement delays in Year 2, the State procured, 
piloted, and released the Edwin Teaching and Learning system in 
Year 3. This aspect of the Edwin platform includes many resources to 
support the standards transition and assessment practices. However, 
educator access was limited in Year 3 because of challenges with 
implementing the Schools Interoperability Framework, which is a 
technology solution for local systems to communicate with State 
systems so data displays reflect real-time changes. Finally, the State 
rolled out Edwin Analytics (formerly the Educator Data Warehouse), 
which provides longitudinal student data to all educators, and in 
Year 3 the State added the Early Warning Indicator System and 
Postsecondary Enrollment and Outcomes reports. 


‘The State also made progress in supporting Race to the Top 
participating LEAs as they implemented new educator evaluation 
systems in Years 2 and 3. In Year 2, the State released a model 
evaluation system and guidance for LEAs developing their own 
systems. Following a 21-school pilot of new evaluation systems in 
nine Level 4 districts in SY 2011-2012, most other Race to the Top 
participating LEAs implemented the professional practices component 
in Year 3, or SY 2012-2013.’ Based on feedback from LEA 


7 Level 3 and Level 4 districts are defined as follows: Level 3 are districts with one 
or more schools among the lowest-performing 20 percent based on quantitative 
indicators; Level 4 are districts identified by quantitative and qualitative indicators 
through a district review. 
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stakeholders, ESE determined that the State was not fully prepared to 
implement student growth measures and that this component would 
not be part of evaluation system implementation in Year 3. 


Over Years 1 through 3 of the grant period, Massachusetts executed its 
plans to build capacity and raise student achievement in its lowest- 
performing schools and districts. Forty-four schools planned and 
began implementing intervention models during SYs 2011-2012 and 
2012-2013.8 ESE led efforts to support schools and LEAs identified as 
low-performing under the State’s accountability system, called Level 3, 
4, or 5 schools and districts (for detail, see Turning Around the Lowest- 
Achieving Schools). In Year 2, the State vetted and selected 24 Priority 
Partners with track records of success in school improvement and 
supported LEA efforts to engage with these partners in implementing 
local school improvement plans. In addition, during Years 2 and 3, 
the State began and expanded programs targeted for educators and 
students in low-performing schools with 110 teachers in Turnaround 
Teacher Teams, 10 leaders in Turnaround Leader Teams, and seven 
LEAs implementing school-based Wraparound Zone plans to address 
out-of-school needs of at-risk students. 


State Year 4 summary 


Accomplishments 


In Year 4, Massachusetts continued to refine its use of the Delivery 
process to track project implementation within ESE, gather data on 
progress, and share updates with senior leadership. This approach 
was effective in identifying projects that were off track and generating 
solutions as well as better integrating related initiatives, such as the 
standards transition with evaluation system implementation. The 
State used external evaluation findings and feedback from the field 

to inform State-level implementation, for example, by providing 
additional supports to LEAs for measuring student learning using 
district-determined measures as part of new evaluation systems. 


‘The State’s college- and career-readiness initiatives included multiple 
strategies for supporting educators in meeting the demands of new 
standards and raising standards for students. In SY 2013-2014, the 
State developed and made available a variety of instructional resources, 


8 Race to the Top States’ plans include supporting their LEAs in turning around 
the lowest-achieving schools by implementing one of the four school intervention 
models: 


Turnaround model: Replace the principal and rehire no more than 50 percent of 
the staff and grant the principal sufficient operational flexibility (including in staffing, 
calendars/time and budgeting) to fully implement a comprehensive approach to 
substantially improve student outcomes. 


Restart model: Convert a school or close and reopen it under a charter school 
operator, a charter management organization, or an education management 
organization that has been selected through a rigorous review process. 


School closure: Close a school and enroll the students who attended that school 
in other schools in the district that are higher achieving. 


Transformation model: Implement each of the following strategies: (1) replace 
the principal and take steps to increase teacher and school leader effectiveness, 
(2) institute comprehensive instructional reforms, (3) increase learning time and 
create community-oriented schools, and (4) provide operational flexibility and 
sustained support. 


standards-aligned assessments, and curriculum documents to support 
educators in the transition to new standards. State-developed model 
curricula cover 95 percent of grades and core academic subjects, which 
exceeds the State's goal. Many educators can access these resources 
through the Edwin Teaching and Learning platform. In addition, after 
delays, teachers accessed pre-Advanced Placement course training 

in Year 4 to offer middle school students rigorous coursework that 
prepares them for Advanced Placement courses in high school. The 
State reported that it would not pursue MassCore as the statewide 
graduation standard since most districts adopt their own graduation 
requirements.” The State reported that over the last five years the 
number of districts adopting graduation requirements similar to 
MassCore is increasing. Forty-seven Innovation schools and six STEM 
Early College High School (ECHS) programs continued to serve 
students in school settings with greater flexibility and STEM-focused 
themes, respectively. 


In the area of Great Teachers and Leaders, Massachusetts LEAs made 
progress in implementing the professional practices component of 
new evaluation systems. The State provided guidance documents and 
technical assistance to LEAs to support local implementation. Teachers 
in all Race to the Top participating LEAs and half of teachers in non- 
participating LEAs received Summative Performance Ratings at the 
end of SY 2013-2014. These ratings were based on a cycle of classroom 
observations and feedback meetings rooted in four performance 
standards: curriculum, planning and assessment; teaching all students; 
family and community engagement; and professional culture. In 
addition, all LEAs developed and executed plans to pilot measures 

of student learning with educators. Finally, the State continued to 
support initiatives for specific education professionals, including a 
superintendent induction program, a performance assessment for 
leaders, and planning for a teacher licensure performance assessment. 
In addition, the State made progress in preparing to implement new 
educator preparation program approval standards. 


ESE continued to support schools identified as low performing 
through site visits and data analysis. These schools worked with 
ESE-vetted preferred providers, project implementation managers, and 
District Plan Managers from the State to strengthen human resource 
management. Seven schools continued to provide wraparound services 
for students, such as health services and social workers. As of fall 
2013, Level 4 schools demonstrated progress that resulted in 14 of 

34 schools exiting Level 4 status. In addition, the State’s one Level 5 
district showed significant gains in student achievement. (For more 
information on how Massachusetts classifies low-performing schools 
and districts, see Turning Around the Lowest-Achieving School). 


Challenges 


In Year 4, the State continued to face challenges in meeting goals 
related to some of its data systems work. While the State rolled out 


®° MassCore requires four years each of English and mathematics; three years each 
of lab-based science and history/social studies; two years of the same foreign 
language; one year of the arts; five additional “core” courses; physical education; 
and additional opportunities, such as Advanced Placement courses, dual 
enrollment, or a senior project. 
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Executive Summary 


the Edwin Teaching and Learning platform statewide, LEA access was 
limited by their ability to comply with the Schools Interoperability 
Framework, which allows for real-time data exchange and display. 
Delays in implementing the interoperability framework have limited 
the period of time in which LEAs can use the system to support local 
instruction and student assessment. In addition, the State was unable 
to make progress on projects intended to support educator use of 
data to inform instruction due to delays and vendor challenges. The 
State’s adjusted approach will extend into SY 2014-2015, reducing 
the time in which educators can engage with the training with grant 
supports. This work is integral to the State’s plans as it complements 
educator work in the standards transition and potential professional 
development needs. 


While the State made progress in implementing the professional 
practice component of the new evaluation system, the State is delayed 
in fully implementing the student growth component. In response 

to feedback from the field, the State determined in Year 4 that it 
would not fully implement the Student Impact Rating portion of the 
evaluation system in SY 2014-2015. ESE continues to make student 
growth percentile data available to educators through Edwin Analytics. 


Looking ahead 


Massachusetts will continue multiple critical work streams during the 
no-cost extension period of its Race to the Top grant in SY 2014-2015, 
or Year 5. Approximately 20 percent of LEAs that accessed grant funds 
will continue into Year 5 supporting local efforts in curriculum work, 
technology investments, and teacher trainings, among other things. 
‘The State plans to maintain grant supports for participating LEAs 

and within ESE through SY 2014-2015. Across project areas and 
consistent with previous years, the State’s work in Year 5 is intended to 
provide tools and resources that are available to LEAs well beyond the 
grant period. 


In Year 5, the State will maintain the myriad resources for educators 
transitioning to new standards and assessments. ESE intends to 
continue expanding its video library of Massachusetts educators 


demonstrating classroom best practice in teaching the new standards. 
‘The State also plans to enhance content currently in the Edwin 
Teaching and Learning system, continue tagging items in the interim 
and formative assessment bank, vet additional locally developed items, 
and provide guidance on how best to use assessment items. Due to 
delays in Year 4, the State will continue to provide pre-Advanced 
Placement training for teachers in Year 5. 


In SY 2014-2015, ESE plans for every participating LEA to become 
compliant with the interoperability framework, which will enable 
many more educators to use the Edwin Teaching & Learning 

system’s assessment and instructional resources. To meet this critical 
milestone the State will continue to provide technical assistance to 
LEAs implementing the interoperability framework and will begin to 
gather statewide data elements from LEAs that are able to transmit 
data in real-time. Given challenges during the grant period to provide 
educators with professional development on using data to inform 
instruction, the State will develop modules for educators to use when 
accessing data currently available in the Edwin platform. In addition, 
the State will provide professional development for school teams in 
some Level 3 schools that did not previously receive targeted supports. 


Participating LEAs will be in their third year of educator evaluation 
system implementation in SY 2014-2015 and all other LEAs will be 
in their second year. As LEAs continue to phase in implementation, 
ESE plans to continue its supports including guidance, trainings, and 
data collection and analysis. In Year 5, ESE will pilot and implement 
new teacher and leader performance assessments. In addition, the 
State will continue to provide a range of professional development and 
curriculum resources for teachers with English learner students and 
support staff working with these teachers. 


In Year 5, schools and districts identified as low performing will 
continue to work with vetted partners to provide services related to 
intervention implementation. Teacher and leader turnaround teams 
will also continue to convene. In addition, the State will host seminars 
to build upon local successes in implementing Wraparound Zones and 
share best practices throughout the State. 
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Race to the Top States are developing a comprehensive and coherent approach to education reform. 
This involves creating plans to build strong statewide capacity to implement, scale up, and sustain the 
reforms initiated by the Race to the Top grant program. 


Building capacity to support LEAs 


ESE’s OPR continued to oversee and track the State’s progress 

in implementing Race to the Top projects through the Delivery 

Unit. For the State’s highest priority projects the Delivery process 
focuses on progress against timelines and quality of implementation. 
Project managers meet once a month and project sponsors meet 

every three weeks to track progress. In addition, the Commissioner 
continued to conduct monthly stocktakes with project teams to 
review progress against metrics. In Year 4, the Delivery Unit combined 
routines for the educator effectiveness and curriculum and instruction 
work streams, which has increased collaboration between staff 
working on these initiatives. For example, staff collaborated on the 
Professional Practice Innovation Grants project and held a joint 
statewide conference. 


‘The State participated in the RSN Sustainability Workgroup 
throughout Year 4 to assess options for sustaining priority reforms 
beyond the grant period. Massachusetts’ approach was featured in two 


briefs, “Performance Management: Setting Outcomes and Strategies 
to Improve Student Achievement,” and “Performance Management: 


Ensuring Accountability for Results.”!” 


Support and accountability for LEAs 


In Year 4, ESE continued its data collection and analysis and oversight 
routines for LEAs implementing Race to the Top work. LEAs reported 
their progress on their performance measures and expenditures at the 
end of SY 2012-2013, which informed ESE supports for SY 2013- 
2014. District and School Assistance Center (DSAC) staff or program- 
specific ESE staff followed up with 24 LEAs for targeted assistance. In 
addition, the State provided differentiated supports to some LEAs. For 
example, large, urban LEAs have ESE liaisons as their main points of 
contact, and Level 3 and 4 districts receive technical assistance from 
DSACs and Readiness Centers. 


’° Publications are available at https://rtt.grads360.org/?p=RTT#communities/sea- 
capacity-building. 
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State Success Factors 


LEA participation 


As depicted in the graphs below, as of August 11, 2014, Massachusetts reported 233 participating LEAs. This represents approximately 69 percent 
of the State’s kindergarten through twelfth grade (K-12) students and approximately 85 percent of its students in poverty. 


LEAs participating 
in Massachusetts’ 
Race to the Top plan 


286,954 
\ 


ia Participating LEAs (#) 
BB Other LEAs (#) 


K-12 students in LEAs 
participating in Massachusetts’ 
Race to the Top plan 


640,596 


4 


|| K-12 students (#) in participating LEAs 
& K-12 students (#) in other LEAs 


Students in poverty in LEAs 
participating in Massachusetts’ 
Race to the Top plan 


307,783 


fea Students in poverty (#) in participating LEAs 
it Students in poverty (#) in other LEAs 


The number of K-12 students and number of students in poverty statewide are calculated using pre-release data from the National Center for Education 
Statistics’ (NCES) Common Core of Data (CCD). Students in poverty statewide comes from the CCD measure of the number of students eligible for free 
or reduced price lunch subsidy (commonly used as a proxy for the number of students who are economically disadvantaged in a school) under the 
U.S. Department of Agriculture’s National School Lunch Program. The students in poverty statewide and number of K-12 students statewide counts are 
aggregations of school-level counts summed to State-level counts. Statistical procedures were applied systematically by CCD to these data to prevent 
potential disclosure of information about individual students as well as for data quality assurance; consequently State-level counts may differ from those 
originally reported by the State. Please note that these data are considered to be preliminary as of August 11, 2014. 


For State-reported context, please refer to the Race to the Top APR at www.rtt-apr.us. 


Stakeholder engagement 


Throughout Year 4, Massachusetts continued to communicate with 
educators in the field and with key education stakeholders to seek 
feedback and promote awareness of the State’s Race to the Top 

work. As in previous grant years, the State created a Year 3 report 
detailing Race to the Top implementation and shared it with a 

variety of stakeholder groups, including local superintendents, union 
representatives, the State legislature, and the Board of Elementary and 
Secondary Education (BESE). 


‘The State continued to provide teachers and leaders with 
opportunities to interact with their colleagues and share best 
practices. The 2014 Educator Evaluation Spring Convening reached 
over 1,000 educators from over 270 districts and provided an 


opportunity for educators to share best practices and plan for 

SY 2014-2015 implementation. In addition, the State leveraged the 
Educator Leader Cadre to provide more than 50 presentations to over 
3,000 participants throughout the school year. These presentations 
reached a variety of stakeholders at professional association meetings, 
schools, districts, and college campuses. To address questions about 
implementation of CCSS and Partnership for the Assessment of 
College and Career (PARCC) assessments, ESE held communication 
sessions across the State throughout the school year and in spring 
2014 held 11 regional community meetings. The State also made 
available a variety of resources (¢.g., updates, guidance documents, 
frequently asked questions, communications tools) to help 
stakeholders understand PARCC and the field test and to provide 
LEAs and schools with tools to help them communicate with parents, 
students, and other audiences. 
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Continuous improvement 


Throughout Year 4, Massachusetts continued to use its existing 
routines to manage projects, engage stakeholders, and track LEA 
implementation. In addition, the Delivery process and stocktake 
meetings supported the agency’s ability to monitor progress 

and quality of implementation and use implementation data to 
update leadership, elevate issues, and execute responses. ESE used 
performance measure reporting and amendment processes to 
determine whether LEAs are on track and to inform differentiated 
supports. State project staff also kept ESE up to date on local 
implementation challenges and successes. These routines are also 
informing which investments and supports to continue beyond the 
grant period. 


Massachusetts’ collaboration with external evaluators has allowed it 
to establish feedback loops to assess its work and identify promising 
practices. In Year 4, these external evaluations informed and validated 
ESE’s work. For example, the implementation study of the educator 
evaluation framework was released in January 2014 to provide 
research findings on early implementation. ESE responded to the 
recommendations by developing plans for additional resources and 
support to LEAs and educators to meet needs in challenge areas, such 
as developing measures of student learning. Feedback from the State's 
external evaluators indicated that ESE’s variety of standards-aligned 
curriculum and instructional resources were useful and of high quality. 
Likewise, evaluation findings related to DSAC data specialist support 
of data routines in Level 4 districts informed the State’s decision to 
continue this investment into SY 2014-2015. Additional evaluation 
reports are expected during fall 2014. 


Successes and challenges 


Through Year 4 of the grant period, Massachusetts effectively 
used the Delivery process to track projects and identify 
projects that are off track. Bringing implementation updates 
to leadership has increased communication and integration of 
Race to the Top and other ESE priority work streams, which 
the State reported improved the quality of its LEA supports 
and guidance across project areas. In addition, the State 
demonstrated that it was using findings and recommendations 
from external evaluation reports to drive adjustments in 
implementation and to inform long-term strategic planning. 


While the State continuously improved its project-level management 
and used LEA-level implementation information to inform its 
approach across project areas, it was unclear to what extent that 
information was being used to inform ESE’s differentiated supports 
to LEAs. The State developed tiered levels of intervention support 
for schools and districts identified as low-performing, but higher- 
performing schools and districts may also need supports given the 
scale of reform. 


Entering the no-cost extension period, or Year 5, the State must 
overcome delays and vendor quality issues experienced throughout 
the first four years, particularly in the area of data systems and data 
use professional development. While the State has plans to complete 
this work, these challenges have limited the period of time in which 
the State can develop and LEAs can benefit from the resources and 
systems planned under the grant. 
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State Success Factors 


Student outcomes data 


Proficiency rates on Massachusetts’ ELA assessments were mixed from SY 2010-2011 through SY 2013-2014; however, proficiency rates in tenth 
grade increased notably during this time period. Proficiency rates on Massachusetts’ mathematics assessments remained about the same from SY 
2010-2011 through SY 2013-2014. 
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100% 
90% 
80% 
70% 
60% 
50% 
40% 
30% 
20% 
10% 

0% 


Percent proficient 


Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 10 


Actual: Actual: Actual: Actual: 
SY 2010-2011 SY 2011-2012 SY 2012-2013 SY 2013-2014 


Student proficiency on Massachusetts’ mathematics assessment 
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Preliminary SY 2013-2014 data reported as of: November 10, 2014. 
NOTE: Over the last four years, a number of States adopted new assessments and/or cut scores. 
For State-reported context, please refer to the Race to the Top APR at www.rtt-apr.us. 
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Achievement gaps on ELA assessments in multiple sub-groups decreased slightly from SY 2010-2011 through SY 2013-2014, including the 
White/Black gap. Achievement gaps on mathematics assessments remained about the same from SY 2010-2011 to SY 2013-2014 for most 
sub-groups. However, the achievement gaps on mathematics assessments for the White/Black and White/Hispanic sub-groups decreased notably 
during this time period. 
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Achievement gap on Massachusetts’ mathematics assessment 
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Preliminary SY 2013-2014 data reported as of: November 10, 2014. 
Numbers in the graph represent the gap over four school years between two sub-groups on the State’s ELA and mathematics assessments. 


Achievement gaps were calculated by subtracting the percent of students scoring proficient in the lower-performing sub-group from the percent of 
students scoring proficient in the higher-performing sub-group to get the percentage point difference between the proficiency of the two sub-groups. 


If the achievement gap narrowed between two sub-groups, the line will slope downward. If the achievement gap increased between two sub-groups, 
the line will slope upward. 


NOTE: Over the last four years, a number of States adopted new assessments and/or cut scores. 
For State-reported context, please refer to the Race to the Top APR at www.rtt-apr.us. 
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State Success Factors 


The high school graduation rate in Massachusetts increased slightly each year from SY 2010-2011 to SY 2012-2013. Massachusetts’ college 
enrollment rate increased from 72.0 in SY 2010-2011 to 75.6 percent in SY 2013-2014, exceeding the State’s target of 75 percent. 
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Preliminary SY 2012-2013 data reported as of: September 15, 2014. 
For State-reported context, please refer to the Race to the Top APR at www.rtt-apr.us. 


College enrollment rate 
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Preliminary SY 2013-2014 data reported as of: September 10, 2014. 


The Department provided guidance to States regarding the reporting period for college enrollment. For SY 2013-2014 data, States report on the students 
who graduated from high school in SY 2011-2012 and enrolled in an institution of higher education (IHE). 


For State-reported context, please refer to the Race to the Top APR at www.rtt-apr.us. 
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Standards and Assessments 


Implementing rigorous college- and career-ready standards and assessments that prepare students for 
success in college and career is an integral aspect of education reform in all Race to the Top States. 


Supporting the transition to college- and 
career-ready standards and high-quality 
assessments 


Adopting standards and developing assessments 


The BESE unanimously voted to adopt the CCSS in ELA and 
mathematics in July 2010. In SY 2012-2013, the State fully 
implemented the Massachusetts Curriculum Framework for English 
Language Arts and the Massachusetts Curriculum Framework 

for Mathematics, which include the CCSS and some State- 

specific standards." 


During Year 4, Massachusetts continued its participation in the 
PARCC consortium to develop assessments. ESE staff participated 
in working groups and project management teams for item 
development and assessment administration contracts. Having piloted 
the Technology Readiness Tool in Year 3, the State continued to 
familiarize LEAs and educators with the PARCC assessments in Year 
4, Throughout Year 4, the State continued to leverage the Educator 
Leader Cadre to speak with a variety of stakeholders about the new 
standards and assessments. ‘These stakeholders included professional 
associations, schools, districts, and college campuses. In addition, 
the State made available multiple communication resources to 
support local efforts to help parents, students, and other audiences 
to understand the standards and assessments.'? In spring 2014, 
approximately 81,000 students across over 1,000 schools and over 
340 LEAs participated in pilots of the PARCC performance-based 
assessment and end-of-year assessments. 


The SY 2013-2014 State assessment tested students on the 
Massachusetts Curriculum Frameworks for the second year. In 
November 2013, the BESE endorsed a two-year transition plan to 
the PARCC assessments. Rather than implement PARCC statewide 
in spring 2015, Massachusetts LEAs can choose to administer 

either PARCC or the Massachusetts Comprehensive Assessment 
System (MCAS) in SY 2014-2015. Administering two assessments 

is a violation of section 1111(b)(3)(C)(i) of the Elementary and 
Secondary Education Act, which requires States to administer the 
same assessments to all students to ensure that the learning progress 
of all students is being measured against the same expectations. 
Massachusetts will provide the Department with a detailed plan 

and timeline on how the State plans to administer one assessment 

to all students by SY 2015-2016. The State reported that the 
two-year transition plan will allow educators to refund standards 
implementation and become familiar with online test administration 
procedures before statewide implementation. The BESE determined it 


" States that adopt the Common Core State Standards (CCSS) are permitted to 
augment the CCSS with up to 15 percent of additional content. 


® See http://www.doe.mass.edu/parcc/ for more information. 


would revisit the plan in fall 2015 to determine whether to implement 
PARCC statewide in spring 2016. Massachusetts intends to remain a 
member of the PARCC consortium during the transition period. 


Throughout Year 4 ESE staff participated in the RSN’s Transitions 
Workgroup, which was designed to support States in navigating the 
transition to college- and career-ready standards-aligned classroom 
instruction using new college- and career-ready standards, assessments, 
and evaluations. 


Supporting college- and career-readiness 


Throughout Year 4 Massachusetts continued to support initiatives that 
promote college- and career-readiness for groups of educators and 
students. Though the State’s pre-Advanced Placement training did not 
occur during SY 2013-2014, as planned, due to a vendor issue, the 
State reached 445 middle- and early-high school teachers in summer 
2014. The pre-Advanced Placement trainings for teachers continued 
to be rated highly by participants and increased student access to 
Advanced Placement coursework beginning in middle school grades. 
In addition, the State and LEAs held students to high expectations 
by promoting MassCore as the recommended high school course of 
study. In its application, the State committed to make MassCore the 
required high school course of study for all high school graduates 
statewide. However, historically, graduation requirements have been 
determined locally and the State believes that this autonomy helps 
LEAs set requirements that meet the unique needs of each LEA. Asa 
result, the State adjusted this commitment to instead encourage LEA 
adoption of MassCore through technical assistance and professional 
development.’’ The State reported that the percentage of high school 
graduates that complete the MassCore course of study increased 
from 50 percent in SY 2009-2010 to nearly 70 percent in SY 2012- 
2013. The analysis indicated that 81 percent of graduates in Level 1 
schools complete MassCore, 77.8 percent of graduates among Level 
2 schools completed MassCore, and 41.2 percent of graduates among 
Level 3 and 4 schools completed MassCore."4 In addition, from 2010 
to 2015 MassCore requirements across all high schools increased 
from 11.4 percent to 25.1 percent. The State plans to incorporate 
these findings in its work with LEA leaders during regional college- 
and career-readiness sessions where district leaders will develop 

plans to support and track student course-taking behaviors. 


In Year 4 Massachusetts continued to support the State’s 46 
Innovation Schools implementing innovative school models. 
Serving 17,000 students in the State, these in-district, charter- 
like schools operate with greater autonomy around curriculum, 
staffing, budget, schedule and calendar, professional development, 


18 See amendment letter dated June 26, 2014, at http://www2.ed.gov/programs/ 
racetothetop/amendments/massachusetts-20.pdf. 
™ Massachusetts’ system of accountability places schools and districts on a five 


level scale, ranking the highest performing in Level 1 and the lowest performing in 
Level 5. See http://www.doe.mass.edu/apa/general/. 
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Standards and Assessments 


and policies. Also in Year 4, 37 LEAs continued participation 
in the Massachusetts Model for Comprehensive School 
Counseling project. The model provided a range of planning, 
professional development, and convening opportunities to 
counselors in participating LEAs during SY 2013-2014. 


Assessments 


In Year 4 the State made available online formative and interim 
assessment items to support educators in implementing the 
standards transition in their classrooms. Through Edwin Teaching 
and Learning educators can access the item bank to develop their 
own formative and interim assessments. In addition, educators can 
opt to have their formative assessments reviewed for inclusion in 
the statewide bank. The State also continued to tag and include 
assessment items in the statewide bank. As of spring 2014, the State 
reported that the system included 7,000 ELA, mathematics, and 
science items tagged by standard. In collaboration with other States, 
Massachusetts is working to obtain a perpetual license for up to 
8,000 items to be included in Edwin Teaching and Learning. For 
more information on the Edwin Teaching and Learning system, 
please see Data Systems to Support Instruction. 


Building off work begun in Year 3, the State continued to create 
Massachusetts Performance Assessments of Knowledge and Skills 
(performance assessments), which are large-scale curriculum- 
embedded assessments. During SY 2013-2014 ESE established a 
council composed of educators who reviewed and provided feedback 
on State-developed performance assessments. In spring 2014 the State 
selected performance assessments to pilot in fall 2014 in the following 
grades and subjects: grade two ELA, grade two mathematics, grade 
four history/social studies, and grade seven history/social studies. 


Dissemination of resources and 
professional development 


In Year 4 the State greatly expanded the number and type of 
resources available to educators for implementing new standards 

and assessments. Approximately 100 educators collaborated in fall 
2013 to continue the model curriculum unit development, which 
resulted in 90 model units that are either in development, being 
finalized, or being piloted in classrooms. These model curricula cover 
95 percent of grades and core academic subjects, exceeding the State’s 
goal. Educators used a rubric to ensure alignment and quality of 
each curriculum unit. All units include lesson plans, digital resources, 
and curriculum-embedded performance assessments. Ten units are 
available to educators through Edwin Teaching and Learning, and the 
State reported it will also make them available on the ESE website. 


‘The State also completed and released drafts of 13 of 21 planned 
curriculum maps, which outline how the Massachusetts Curriculum 
Frameworks can be taught over the course of an academic year. The 
State reported that it completed all 21 curriculum maps by September 
2014. In addition, the State is developing resources to support LEAs 
in developing their own curriculum maps. Mapping functionality is 
available in the Edwin Teaching and Learning system. 


In addition to the resources described above, the State worked to 
build a digital library within Edwin Teaching and Learning to provide 
educators with high-quality standards-aligned instructional resources, 
including lesson plans, essays, video, audio, and images. As of October 
2014, the digital library included thousands of instructional resources 
in mathematics, ELA, science, social studies, arts, health and physical 
education, and world languages for pre-K through postsecondary. In 
addition, the State continued to work with WGBH, the State’s public 
television station, to identify, revise, or create resources, tag content, 
and identify appropriate resources to pair with the above-described 
curriculum units. The WGBH videos capture educators engaged in 
and reflecting on model curriculum development. 


Successes and challenges 


During Year 4 Massachusetts made progress in developing a range 

of resources to support educators in the standards and assessments 
transition. Educators have access to 90 model curriculum units, 
curriculum embedded performance assessments, thousands of tagged 
digital resources, and thousands of interim and formative assessment 
items. Students also have opportunities to meet college- and career- 
ready expectations with increasingly rigorous high school courses of 
study and flexible models in Innovation Schools. 


State evaluation findings indicated that standards implementation 
was mixed in SY 2013-2014. In winter 2014, LEAs reported being in 
various stages of creating implementation plans for the Massachusetts 
Curriculum Frameworks. In addition, elementary and middle school 
teachers were more likely than high school teachers to report having 
aligned instructional practice to the new standards. While the State 
has made a number of resources available, it was unclear whether 
they were being widely used as intended at the local level. Challenges 
with accessing the Edwin Teaching and Learning system may have 
impacted educators’ ability to use State-developed resources during 
the transition period. As a result, it will be important for the State to 
respond to issues raised in the standards implementation survey. In 
addition, given the investment in the Edwin Teaching and Learning 
system and its potential to support educator practice it will be 
important for the State to assess implementation to ensure the system 
is used to inform instruction. 
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Data Systems to Support Instruction 


Statewide longitudinal data systems (SLDS) and instructional improvement systems (IIS) enhance the 
ability of States to effectively manage, use, and analyze education data to support instruction. Race to 
the Top States are working to ensure that their data systems are accessible to key stakeholders and 
that the data support educators and decision-makers in their efforts to improve instruction and increase 


student achievement. 


Fully implementing a statewide 
longitudinal data system 


Massachusetts’ Year 4 data system work centered on the Edwin system, 
composed of Edwin Analytics and Edwin Teaching and Learning, and 
the Schools Interoperability Framework, the technology infrastructure 
to exchange data in real-time. In Year 4 the State made progress in 
adding reports to Edwin Analytics, including resources in Edwin 
Teaching and Learning, and supporting LEAs in implementing the 
Schools Interoperability Framework. 


The Schools Interoperability Framework allows local student 
information systems to share data with State systems in real time 

and enables educator access to the Edwin Teaching and Learning 
platform. Educators in a district cannot access the benefits of Edwin 
Teaching and Learning until their LEA’s data systems comply with 
the interoperability framework. Once implemented, the Schools 
Interoperability Framework reduces reporting burden on LEAs by 
connecting local student information systems to State-level systems 
making updates in real-time (¢.g., daily) through automated data 
pushes. State systems can then securely draw relevant data into 
multiple other systems, for example to report student outcomes 

and attendance or to link teachers to data within Edwin Analytics. 
Without the interoperability framework, LEAs must format data from 
their student information systems and send files to ESE during defined 


periods of time. 


After significant vendor challenges in Years 2 and 3, the State 
determined a sustainable solution for Year 4 implementation and 
further supported Race to the Top participating LEAs to complete 

the interoperability framework. In SY 2013-2014, the State made 
progress onboarding more LEAs and these LEAs were able to transmit 
data via the Schools Interoperability Framework. However, the 
project continued to be delayed due to local student information 
system vendor certification delays, vendor delays in connecting to the 
interoperability framework, and LEA capacity challenges. The State 
reported that 60 percent of Race to the Top participating LEAs were 
enabled and could transmit data by the end of Year 4, and 108, or 
about 46 percent, certified their data via the Schools Interoperability 
Framework. This fell short of the State’s Year 3 goal of having all 
participating LEAs transmit data through the interoperable framework 
by Year 4. The State reported that State-level technology issues have 
been resolved and that it completed certification protocols with all 


student information system vendors in the State. Going forward, 
increasing the scale of compliance will depend on ESE coordinating 
and communicating with LEAs and LEAs working with their student 
information system vendors. The Department approved the State’s 
request to support statewide Schools Interoperability Framework 
implementation, including LEAs not participating in Race to the Top, 
during SY 2014-2015. 


Throughout Year 4, Massachusetts developed and released new 
functionality to the Edwin Analytics system to expand educator access 
to multiple kinds of data.'* Edwin Analytics remains the State’s main 
tool for reporting and analyzing historic longitudinal student data 
from pre-kindergarten through postsecondary education programs. 
In Year 4 the State reported that the Early Warning Indicator 

System (EWIS) was the most frequently used report. EWIS reports 
identify student in grades 1-12 who may be at risk of missing critical 
educational targets (¢.¢., graduating from high school). As of June 
2014, the State added reports on district finance data that provide 
information on local-level expenditures on program and budget items 
to support local capacity to evaluate investments and budgets. In 
addition, the State added functionality to release data to educator 
preparation programs to inform their Preparation Program Profiles 
(see Great Teachers and Leaders for more detail). 


‘The State reported that investments in Edwin Analytics’ system 
performance have reduced the occurrence of efforts and increased 
usage. In May 2014, Massachusetts reported a 90 percent increase 
in the number of reports run without performance issues. As of 
April 2014, 62,000 more reports had been generated in the first few 
months of 2014 than had been generated in all of 2013. Finally, the 
average number of users during SY 2013-2014 was higher than in 
SY 2012-2013. 


During Year 4, Massachusetts also made progress in releasing and 
expanding use of the Edwin Teaching and Learning system, including 
many of the curriculum materials, instructional resources, and 
formative assessment items described in Standards and Assessments. 
However, widespread use was limited due to delays with implementing 
the Schools Interoperability Framework (see above). Educators are 
able to add items if they access resources through Edwin Teaching and 
Learning. In addition, educators can create, score, and report interim 
assessment results, which provide shorter cycle information about 
student progress in meeting standards. 


*® Edwin Analytics was formerly known as the Educator Data Warehouse. 
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Accessing and using State data 


As described above, the State makes resources related to the standards 
and assessments transition available through the Edwin Teaching 
and Learning system. Having jointly procured the system with Ohio 
in December 2012 and piloted it with 34 LEAs in spring 2013, the 
State released Edwin Teaching and Learning to all participating LEAs 
in SY 2013-2014. Though the State planned to release full system 
functionality at the start of the school year, the State reported that 
only 80 percent of the system's functionality was available. However, 
all planned functionality was available to LEAs by the start of 

SY 2014-2015. Use of the system in SY 2013-2014 was slower than 
expected because of vendor delays and LEA-level capacity challenges 
with implementing the interoperability framework, which is required 
prior to educator access. At the start of SY 2013-2014, 166 Race to 
the Top participating LEAs committed to use the system, but just 82 
LEAs were compliant with the interoperability framework and able 
to fully utilize the system as of June 2014. Some LEAs are phasing 

in implementation of the components of the Edwin platform with 
teachers over multiple school years. 


To expand awareness of the resources available in Edwin Teaching 

and Learning, the State held “roadshow” events during fall 2013. At 
these events, ESE staff introduced features of the system and educators 
engaged with the system through hands-on demonstrations. Over 
1,000 educators participated in the sessions. In addition, curriculum 
mapping training included demonstrations of how to use mapping 
tools in Edwin Teaching and Learning. The State also posted three 
Edwin Analytics videos to support teachers in creating reports and 
dashboards of student-, classroom-, and school-level data that can 
inform instructional decision-making. 


Using data to improve instruction 


In Year 4, DSAC data specialists served 56 Level 3 and 4 LEAs across 
the State with training and support on effective data practices. For 
example, DSAC data specialists supported LEAs in developing district- 
or school-level structures, like data teams, to analyze and share data; 
using data to identify areas for school and district improvement 
planning; improving classroom instruction based on data; and, using 
data as part of a continuous cycle of improvement. 


‘The State worked with a vendor to develop online professional 
development courses to support educator use of data to inform 
instruction, but encountered challenges and ongoing delays. In 

Year 3, the State identified a vendor to develop four courses, totaling 
65 hours, for educators and school and district administrators. The 


Data Systems to Support Instruction 


State and vendor developed two courses, Foundations in Data 
Literacy and Assessment Literacy, but only piloted one course. 
During Year 4, the State reported challenges with the quality of the 
vendor’s “Data in Action” deliverables and concerns about whether 
educators would engage with the courses. As a result, during summer 
2014, the State developed a different approach to supporting educator 
use of data through SY 2014-2015. The State plans to procure a 
vendor to develop online modules within existing State data systems 
to assist educators in using reports and dashboards to inform 
instruction. In addition, the State plans to provide professional 
development on effective data use for a group of schools that do not 
have DSAC supports. 


Successes and challenges 


In Year 4, Massachusetts’ data system work made progress 

but challenges remain. The State resolved some of the Schools 
Interoperability Framework implementation barriers and expanded 
the functionality in Edwin Analytics. Usage of Edwin Analytics 
appears to be increasing and providing LEA and school leaders with 
important historic student data. A variety of curriculum, instructional, 
and assessment materials are available to educators through the Edwin 
Teaching and Learning system, but use of this system depends on 
whether the LEA is compliant with the interoperability framework. 
With 108 LEAs, or 45 percent, of participating LEAs able to transmit 
data via the interoperability framework by the end of Year 4, the State 
will have to increase supports in fall 2014 to ensure educators can 

use the resources and access real-time data in Edwin Teaching and 
Learning and Edwin Analytics. 


DSAC data specialists continued to provide valuable supports to Level 
3 and 4 LEAs in the State. Preliminary evaluation findings indicated 
that DSAC supports helped educators to analyze data, establish goals, 
and use assessment data to adjust instructional practice to better 

meet student needs. Since Year 2, 9,700 educators have participated 
in DSAC training and another 2,000 have been trained on using the 
EWIS. District and school leaders also indicated that they saw more 
evidence of teachers using data to inform curriculum and instructional 
practices, for example, by observing differentiated instruction, re- 
teaching, or identifying interventions. The State’s efforts to provide 
additional professional development on effective data use through 

the “Data in Action” initiative remained delayed in Year 4 and the 
State began to plan an alternative approach for SY 2014-2015. While 
the State successfully released foundational data systems and provided 
training through DSACs and on the EWIS, more training on effective 
use of that data is still needed. 
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Great Teachers and Leaders 


Race to the Top States are developing comprehensive systems of educator effectiveness by supporting 
high-quality pathways for aspiring teachers and principals, ensuring equitable access to effective teachers 
and principals, improving the effectiveness of teacher and principal preparation programs, and providing 
effective supports to all educators. As part of these efforts, Race to the Top States are designing and 
implementing rigorous, transparent, and fair evaluation systems for teachers and principals; conducting 
annual evaluations that include timely and constructive feedback; and using evaluation information to 
inform professional development, compensation, promotion, retention, and tenure decisions. 


Improving teacher and principal 
effectiveness based on performance 


Educator evaluation 


In Year 4, Massachusetts built upon its work in Years 1 through 3 

to introduce each component of the new evaluation systems being 
implemented in all LEAs. About 25 percent of Massachusetts LEAs 
adopted the Massachusetts Model System for Educator Evaluation 
and the rest adapted the State model to meet local needs while 

still complying with State regulations. Every evaluation system 

will ultimately result in a Summative Performance rating based on 
classroom observations and a Student Impact Rating based on MCAS 
student growth percentiles and/or district-determined measures. '° 
‘There are four possible Summative Performance Ratings: exemplary, 
proficient, needs improvement, or unsatisfactory; there are three 
possible Student Impact Ratings: high, moderate, or low. Although 
Massachusetts initially expected all Race to the Top participating 
LEAs to fully implement all components of their educator evaluation 
systems in Year 3, ESE deferred implementation of the Student 
Impact Rating based on stakeholder feedback. 


During SY 2013-2014, Race to the Top participating LEAs 
implemented local evaluation systems with all educators and the 
remaining LEAs implemented their locally adopted evaluation system 
with at least 50 percent of educators. Educators implemented district- 
determined measures and received MCAS student growth percentile 
data, but did not receive the formal Student Impact Rating described 
above. The State reported that nearly a third of LEAs planned to pilot 
more than the required five district-determined measures of student 
8. A Rating of Impact on Student Learning is based on trends and patterns in 
student learning, growth, and achievement. In order for a Rating of Impact on 
Student Learning to be given, educators must have two years of data from at 
least two State or district-wide growth measures. State regulations require that 
Massachusetts Comprehensive Assessment System (MCAS) student growth 
percentiles be used as one measure where available (/.e., for teachers of tested 
grades and subjects). For those educators for whom MCAS student growth 
percentiles are not available, LEAs must develop district-determined measures 
to serve as measures of growth. According to the State, district-determined 
measures are measures of student learning, growth, and achievement related to 
the Massachusetts Curriculum Frameworks, Massachusetts Vocational Technical 
Education Frameworks, or other relevant frameworks, that are comparable across 
grade or subject levels district-wide. These measures may include, but shall 
not be limited to: portfolios, approved commercial assessments and district- 
developed pre- and post-unit and course assessments, and capstone projects. 
On the basis of these data, educators receive a low, moderate, or high Rating of 
Impact on Student Learning. 


learning, indicating their commitment to practice and refine this 


component of the evaluation system. '7 


‘The State released additional guidance documents and trainings to 
support statewide implementation of evaluation systems in 

SY 2013-2014. Many of these supports were developed in response 
to feedback from preliminary findings of the State’s independent 
evaluator, conducted in fall 2012. In particular, ESE tailored new 
supports to meet a need for evaluation systems to integrate with 
other reform initiatives and a need for more training and guidance 
on goal-setting and evidence collection. The State released new or 
updated “Quick Reference Guide” documents for educators on 
educator evaluation and its intersection with professional development 
and with the Massachusetts Curriculum Frameworks. In addition, in 
fall 2013, ESE released guidance on how to use current assessments 
in district-determined measures to measure student growth that 
included using model curriculum units with curriculum-embedded 
performance assessments. In fall 2013, the State awarded Professional 
Practice Innovation grants to six LEAs to support local efforts to 
integrate evaluation implementation. For example, one group of 
LEAs worked together to develop Common Course Objectives and 
accompanying assessments using science, physical education, art, and 
music curriculum frameworks. These Common Course Objectives will 
be used for district-determined measures. In another LEA, educators 
created a series of videos of effective practices, including creation 

and refinement of standards-based lessons, high-quality instruction 
of those lessons, and educator observation and feedback. The State 
also released a SMART goal development tool to support educator 
development of goal statements.'* Other training modules and 
guidance documents released by the State in previous grant years 
continue to be available on ESE’s website. 


In addition to the supports described above, ESE provided other 
opportunities for LEAs to improve their practice in implementing 
new evaluation systems. The State made educator evaluation 
implementation surveys available to LEAs for teachers and principals/ 
administrators. Administering these surveys is optional for LEAs and 
ESE does not collect survey data. In addition, ESE supported three 


The State reported that 96 percent of LEAs submitted these plans to the 
Massachusetts Department of Elementary and Secondary Education (ESE); 
82 percent of submitted plans exceeded the State’s minimum requirement for 
school year (SY) 2013-2014 implementation. 


8 SMART goals are specific, measurable, attainable, realistic, and timely goals. 
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LEAs through educator evaluation technology innovation grants to 
explore ways of enhancing evaluation implementation through the 
use of technology applications. Lastly, the 2014 Educator Evaluation 
Spring Convening reached over 1,000 educators from over 270 
districts and provided an opportunity for educators to share best 
practices and plan for SY 2014-2015 implementation. 


The State, in collaboration with the Massachusetts Association 

of School Superintendents, continued its support of the New 
Superintendent Induction Program in Year 4. The three-year program 
is designed to increase superintendent effectiveness and improve 
leadership abilities. As of May 2014, 11 new superintendents 


participated in the program in addition to the 68 already participating. 


Ina July 2014 evaluation report, participating superintendents 
reported that the program had changed their thinking about their 
role and that they spent most of their time focused on instructional 
improvement activities. Due to the success of the program, the 
State and the Massachusetts Association of School Superintendents 
recruited a fifth cohort to begin the program in SY 2014-2015. 


‘Throughout Year 4, Massachusetts participated in RSN convenings 
and webinars with colleagues in other Race to the Top States related 
to evaluation system implementation. In particular, Massachusetts 
participated in a convening on improving the accuracy and efficacy 
of evaluation systems in which States worked to identify patterns, 
outliers, and concerns from available effectiveness data and then 
develop an action plan to address issues. In addition, the RSN 
supported Massachusetts in identifying best practices from other 
States engaged in work similar to the State’s district-determined 
measures work. 


Human capital management 


In Year 4, Massachusetts made progress in developing performance 
assessments that align to new educator preparation and licensure 
standards passed in December 2011, in addition to other human 
resources efforts. During SY 2013-2014, 120 principal candidates 
piloted the Massachusetts Performance Assessment for Leaders (MA- 
PAL), which is intended to assess candidates for principal licensure 
based on leadership knowledge and skills. MA-PAL is composed of 
four performance assessment tasks designed to authentically reflect 
work of school leaders. Candidates in the pilot provided feedback 
on their experience with two performance tasks and trained scorers 
provided feedback on their experience using rubrics to evaluate the 
tasks. An analysis of task outcomes, feedback, and content informed 
task revisions and the field test planned for SY 2014-2015. All 
principal candidates will participate in the field test and the State will 
analyze outcomes to ensure the assessment meets its purpose. 


Following delays earlier in the grant period, Massachusetts also 
made progress developing a teacher leader performance assessment 
in Year 4. Throughout SY 2013-2014, Massachusetts worked with 


Massachusetts Performance Assessment for 
Leaders (MA-PAL) 


During SY 2013-2014 Massachusetts field tested a new leader 
licensure assessment rooted in the following four performance 
tasks: 


Task 1 — Leadership through a vision for high student achievement 
Task 2 — Instructional leadership for a professional learning culture 


Task 3 — Leadership in observing, assessing, and supporting 
individual teacher effectiveness 


Task 4 — Leadership for family engagement and community 
involvement 


Candidates demonstrate their success in these tasks by developing 
a data-driven school vision and improvement plan, working with 
small groups of teachers, completing a teacher observation cycle, 
and developing a family engagement proposal. For more information, 
visit http://ma-pal.com/. 


other States through a Council of Chief State School Officers grant 
to conduct research and develop recommendations around whether 
to adopt, adapt, or build a teacher performance assessment.’ The 
group met six times throughout the year and developed five 

areas of recommendations for a fall 2015 pilot: development of a 
comprehensive assessment system, observation protocol, student 
feedback surveys, portfolios, and simulations. Based on the 
recommendations and the Commissioner’s decision, the State 
intends to revise its teacher licensure policies and implement a 
teacher licensure performance assessment in fall 2015. In addition, 
Massachusetts began to execute against a revised approach related to 
teacher career ladder endorsements. ‘The State worked with two LEAs 
(Springfield and Lawrence) to document and analyze their locally 
developed career ladder differentiation initiatives. A policy brief 
currently in development will be distributed to LEAs to share lessons 
learned from these initiatives. The State plans to provide technical 
assistance to LEAs interested in developing local career ladder 
initiatives for teachers. 


Following the human resources pilot program in Years 2 and 3 of the 
grant period, the State developed a toolkit with promising practices 
and exemplars from 30 LEAs. The toolkit includes materials used to 
implement the practices and self-reported descriptions of the practice 
from LEA leaders. The State is working with its technical assistance 
partner to evaluate the experience of the three urban LEAs that 
participated in the pilot program. It is unclear whether the State will 
provide additional supports to LEAs to improve their human resources 
practices, as was planned. 


8 The purpose of this grant is to allow for States to work together to influence the 
development of innovative licensure performance assessments that are aligned 
to the revised licensure standards and include multiple measures of educators’ 
ability to perform, including the potential to impact student achievement and 
growth. 
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Ensuring equitable access to effective 
teachers and principals 


The State’s work to increase equitable access to effective educators drew 
upon various strategies to meet educator training and development 
needs once in the classroom. These strategies included a survey 

of teacher feedback on educational issues, a recruitment portal, 
mentorship programs, and certification programs. 


In Year 4, the State drew upon various data sources related to educator 
performance and perceptions to inform State and district planning 

of human capital practices. Following SY 2013-2014, 91 percent of 
participating LEAs submitted educator ratings on the Summative 
Performance Rating component. This data will inform ESE’s 
monitoring of the distribution of effective educators. In addition, the 
State administered the Teaching, Empowering, Leading, and Learning 
in Massachusetts (TELL Mass) survey for the second year and reached 
approximately 48 percent of educators statewide. The survey provided 
educator feedback on educational issues such as school facilities and 
resources, community engagement and support, and instructional 
resources. The State reported that LEAs used this data to inform 
professional development planning and set school goals. In comparing 
2012 results with 2014 results, Massachusetts educators reported the 
greatest increase in positive perception on items related to community 
support and involvement. 


In Year 4 the State worked to publicize the aMAzing Teachers website 
and increase its usage following its summer 2013 release.” The 
website provides information about teacher and leader licensure 
requirements, teaching opportunities in low-performing schools, 
educator preparation programs, and job opportunities. Job candidates 
can search and apply for jobs, and recruiters can post positions, search 
among candidates, and determine if they hold valid licenses. The State 
reported that the website has experienced a steady increase in usage in 
areas such as the number of job postings. 


‘The State continued to expand the supply of highly-effective teachers 
and leaders through mentoring, leadership, and training programs. 
By the end of SY 2013-2014, 206 mentors from the first two cohorts 
of Project SUCCESS completed their training and another 206 
mentors were participating in the third cohort. Project SUCCESS 
helps high-need school districts retain new teachers by improving 
teacher performance through training on best practices. Participants 
engage in a nine-month mentoring program with leadership sessions 
and online course modules and go on to become mentor-leaders to 
provide training to future cohorts. Massachusetts plans to make 

all resources available on its website for LEAs to use for their own 
programs. The State also continued to provide a subsidy to educators 
in high-need schools to obtain National Board certification, with 76 
educators achieving certification in Year 4. The subsidy more than 
doubled the number of educators pursuing certification, with just 42 
candidates applying in SY 2013-2014 and about 30 teachers expected 
to become certified. Massachusetts is unlikely to reach its target of 


2° For more information, see http://www.doe.mass.edu/amazingeducators. 


200 educators in high-need schools obtaining certification by the 
end of the grant period. The State also continued its partnership with 
the National Institute for School Leaders with over 730 participants 
completing the district leadership program during the grant period. 
‘The training in this program focuses on research-based principles of 
learning, teaching, and curriculum and building capacity to turn 
learning theory into practice. 


A major component of the State’s approach to equity has been 
supporting teachers with Sheltered English Immersion training to 
ensure the academic success of English learner students, the fastest 
growing group of students in the State. ESE operates a waiver 
program that allowed educators to complete licensure requirements for 
teaching English learners while working in a K-12 school. However, 
new regulations adopted in 2012 require all core academic teachers 

to obtain a Sheltered English Immersion license, resulting in fewer 
teachers applying for waivers in 2013. To better support core academic 
teachers with English learner students in light of the new requirements, 
the State changed its approach. During Year 4 the State determined 
that online professional learning networks were not an effective 
strategy for providing Sheltered English Immersion collaboration 
opportunities. Rather than use professional learning networks, the 
State is developing videos and other virtual simulations to train up 

to 9,000 teachers, including those who do not currently have English 
learners in their classrooms but may in the future. The State will 

also invest in developing a technology platform to disseminate the 
courses more widely; this platform will eventually house a variety of 
professional development courses. These courses are in addition to 
ongoing efforts to develop curriculum to support teachers teaching 
English learners, courses for literacy and mathematics coaches on 
instructional strategies, and training for teachers that support core 
academic teachers using Sheltered English Immersion strategies. The 
State's work to develop supports for these teachers will continue into 
the no-cost extension period. 


‘The State also continued to track LEAs with teachers seeking licensure 
to teach students with disabilities and reported that LEAs met their 
targets to varying degrees. As of spring 2014, 46 teachers were 
enrolled in courses that will lead to licensure. 


Improving the effectiveness of teacher 
and principal preparation programs 


‘The BESE approved revised program approval regulations in June 
2012 following a development and revisions process in which 
stakeholders provided feedback and input. In Year 4, Massachusetts 
made some progress in implementing new educator preparation 
program approval standards but remains delayed in this area. Having 
completed the information technology infrastructure work in Year 3, 
the State collected data from preparation programs for the first time in 
summer 2013. The State released Preparation Program Profiles, which 
include publicly available data about candidate enrollment, program 
completion, and licensure exam pass rates. In December 2013, the 
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State added employment and retention data to the Profiles. The State 
updated the data elements for each program profile with additional 
candidate cohort data in summer 2014 and will continue to build 
connections between data sets to add data elements to the profiles 

in Year 5. The State did not include measures of student growth in 
the Profiles, as planned, and decided not to do so until those data 
elements are better understood by stakeholders. 


Massachusetts continued to provide technical assistance and outreach 
to preparation programs regarding the revised regulations through 
individual meetings, conferences, and a quarterly newsletter. The State 
published revised “Guidelines for Program Approval” in July 2013 
and made progress in Year 4 to develop new aspects of the educator 
preparation program approval process. The guidelines are based on 
revised regulations, passed in June 2012, that shifted the program 
approval process to be outcome- and evidence-based, expanded data to 
report to the State, and included the publication of an annual report 
with data from each program. ‘The State piloted some elements of the 
regulations with programs during the SY 2013-2014 review cycle, but 
found more time is needed to implement and review new approaches. 
ESE will continue this work into Year 5 with plans to complete 17 
program reviews by June 2015. 


Massachusetts continued to support two teacher preparation program 
awardees that graduated 39 candidates in SY 2013-2014. In addition, 
after not receiving principal preparation program applications that 
met the State’s requirements for funding, in Year 4 the State identified 
three principal programs for funding. These programs will begin 
implementation in Year 5. 


Providing effective support to teachers 
and principals 


‘The State’s work in this area centered on developing standards for 
high-quality professional development to ensure that Massachusetts 
educators have professional learning experiences that improve teacher 
practice and student outcomes. Having finalized the Massachusetts 
Standards for Professional Development in 2012, the State 

worked in Year 4 to create resources and materials to support local 
implementation of the standards.*! The Professional Development 
Evaluation Toolkit and District and School Professional Development 
Planning Guide serve as the two main resources and will be released 
in Year 5 after a taskforce of educators contributes revisions. The 
State plans to provide training on using the Evaluation Toolkit and 
Planning Guide throughout SY 2014-2015, nearly two years later 
than planned. Also in Year 4, a cross-unit team in ESE developed 
recommendations for revising current professional development 
policies and guidelines and began revising the preferred provider 
application. The State is delayed in publishing a provider list, which 
was planned for SY 2012-2013, but plans to begin approving 
providers in fall 2014. 


21 For more detail, visit http://www.doe.mass.edu/pd/standards.html. 


In March 2014, ESE reviewed its professional development offerings 
using an assessment tool that analyzes alignment to the Massachusetts 
Standards for Professional Development and the State educator 
evaluation framework. Initial reports from the review indicated 

that about half of ESE offerings align to all of the standards for 
professional development and that additional awareness among 
developers was needed about expectations for using the standards. 


During SY 2013-2014, a second cohort of over 400 participants from 
20 districts participated in professional learning communities (PLCs) 
intended to drive instructional improvement and increase student 
learning and achievement. The State held two-day Summer Institutes 
in 2013, followed by three days of training during the school year. 
DSAC staff members also participated in the trainings to support 
LEAs and schools to establish and maintain strong PLCs. In order 

to support this work going forward, the State is developing a PLC 
website which will include resources such as case studies, modules, and 
other guidance documents. 


Successes and challenges 


Massachusetts made progress across its Great Teachers and Leaders plans 
in Year 4. In particular, all Massachusetts LEAs implemented aspects 
of the evaluation system that resulted in Summative Performance 
Ratings and piloted district-determined measures to varying degrees. 
‘The State successfully provided a wealth of resources to support local 
implementation at each stage of the evaluation cycle. In addition, 

the State integrated evaluation system guidance with other reform 
efforts, such as the standards transition. The State has not yet provided 
Student Impact Ratings to educators, but districts made progress 

in piloting district-determined measures of student learning for all 
educators. Although the State has supported local implementation, it 
was unclear how the State would gather lessons and challenges from 
SY 2013-2014 implementation to inform differentiated supports in 
the future. This is critical to inform additional educator supports to 
implement the evaluation system consistently and rigorously. 


Many of the State’s projects related to equitable access to teachers 
continued to be on track in SY 2013-2014, though the State missed 
their program completion targets by the end of the four-year grant 
period. The TELL survey provided valuable information about school 
climate, the aMAzing website increased local access to a diversity 

of teacher candidates, and Sheltered English Immersion training 
supported teachers with English Learner students. However, it is 
unclear whether, taken together, these strategies have met the goal of 
ensuring that students in high-minority, high-poverty schools have 
access to effective teachers and leaders. 


‘The State’s Preparation Program Profiles continue to reflect new data 
elements and more cohorts, bringing transparency to the public 

on educator preparation program performance. The State learned 

of the importance of stakeholder awareness and involvement as it 
considered other data elements. In Year 4, the State also made progress 
towards implementing new program approval processes that focus on 
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outcomes and evidence and in supporting new teacher and principal 
preparation programs. 


Lastly, the State successfully finalized toolkits and planning guides 
to support local implementation of new professional development 
standards. Given local efforts to implement reform initiatives like new 


evaluation systems and standards, this guidance may be particularly 
timely in supporting local efforts to be thoughtful about professional 
development investments and plans. However, because of delays it will 
be critical that ESE support local use of these resources. 


Turning Around the Lowest-Achieving Schools 


Race to the Top States are supporting LEAs’ implementation of far-reaching reforms to turn around 
lowest-achieving schools by implementing one of four school intervention models. 


Intervening in the lowest-achieving 
schools 


Implementing intervention models 


In SY 2013-2014, Massachusetts continued identifying the lowest- 
performing schools, working with them to implement intervention 
models, and assessing their progress.” Many schools demonstrated 
student achievement progress resulting in moving from lower- 
performing statuses to higher-performing statuses. Across SYs 2011- 
2012, 2012-2013, and 2013-2014, Massachusetts identified 54 Level 
4 schools. As of fall 2013, the 35 schools identified in 2010 had been 
implementing intervention models for three school years. Of these 
schools, 40 percent exited status, 43 percent remained in status, 

11 percent were designated as Level 5 schools, and 6 percent closed. 
‘The State reported that all 14 schools that exited status met 80 percent 
or more of their annual benchmarks and had conditions in place 

for school effectiveness that led ESE to believe the schools no longer 
needed intensive support and interventions. However, because these 
schools are in Level 4 districts they will continue to receive support 
and monitoring.” For newly identified Level 5 schools, SY 2013- 
2014 served primarily as a planning year as the State worked to assign 
receivers, develop school improvement plans, and communicate with 
stakeholder groups. 


2 ESE identifies persistently lowest-achieving (PLA) schools based on the State’s 
accountability rating system. ESE selects Level 4 schools from among the lowest- 
performing 20 percent of schools in the State based on student achievement 
and growth in English language arts (ELA), mathematics, and science, as well as 
graduation and dropout rates for high schools. Schools that demonstrate the least 
improvement receive the Level 4 designation. In turn, Level 4 schools that fail to 
reach improvement benchmarks after three or more years receive Level 5 status, 
which triggers State governance and requires the Commissioner to convene a 
local stakeholder group, create a turnaround plan that will rapidly improve student 
achievement, direct the superintendent or an appointed receiver to implement the 
plan, and evaluate the school at least annually. Level 4 schools that successfully 
apply for School Improvement Grant funding must implement one of the four 
school intervention models. All schools designated as Level 4 must implement 
a turnaround plan, which needs to be aligned with the turnaround principles 
presented in the State’s approved Elementary and Secondary Education Act 
flexibility request. 


Massachusetts districts are classified at the level of their lowest-performing 
school. Districts are independently eligible for Level 5 designation based on 
a variety of factors. For more detail, visit http://www.doe.mass.edu/apa/sss/ 
turnaround/level5/districts/. 


np 
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In Year 4, Massachusetts continued to track the progress of Level 4 
and 5 schools through site visits to identify strengths and weaknesses 
and provide formative feedback for leaders to identify action steps. 
Based on these interactions, Massachusetts focused their district 
supports on strengthening human resource management, project 
implementation, and governance practices. In these three areas 

the State subsidized partnerships with proven vendors to support 
local implementation of educator evaluation systems and share best 
practices. In addition, the State continued to support District Plan 
Managers for the six Level 4 districts that required significant support 
to implement their improvement plans. 


‘The State’s only Level 5 district, Lawrence, demonstrated significant 
progress based on SY 2012-2013 MCAS results. Since naming a 
receiver in January 2012, the district implemented multiple strategies 
to achieve positive outcomes for students. Namely, the district's 
mathematics scores were the highest ever recorded, a three-year decline 
in ELA proficiency was reversed, the number of Level 1 schools in the 
district doubled, and the number of schools in which student rates of 
growth outperformed the State average nearly tripled. 


Massachusetts expanded the District Governance Support Program 
to 25 additional districts in Year 4. Begun in Year 2 and led by the 
Massachusetts Association of School Committees, the program is 
designed to help school committees focus on student achievement in 
collaboration with superintendents. Twenty-five Level 3 and 4 districts 
engaged with the District Governance Support Program curriculum, 
which includes guidance documents for superintendent and school 
committee roles and responsibilities, trainings on superintendent 
evaluations, and school committee coaching. The State reported 

that the Massachusetts Association of School Committees intends to 
continue supporting the program after the grant period. 


Promoting student achievement in low-performing 
schools 


During Year 4, seven LEAs continued to implement their Wraparound 
Zone initiatives, which are intended to build school and district 
capacity to systematically address barriers to student learning. Started 
in Year 2, Wraparound Zone schools create proactive systems for 
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District-led Wraparound Zones 


The opportunity to implement local approaches to wraparound 
services over three school years revealed important successes 

and lessons learned in addressing students’ non-academic needs. 
Schools implementing the initiative made progress in the areas of 
climate and culture with greater educator awareness, decreased 
discipline referrals, and improved family engagement. Implementing 
schools also described the importance of maintaining district-level 
support of school programs to ensure buy-in and sustainability. Two- 
thirds of the Level 4 schools that began the initiative exited Level 4 
status after SY 2013-2014. Future evaluation efforts will focus 

on the connection between the wraparound zone initiative and 
academic outcomes. While implementation varied among schools 
and progress takes time, the wraparound zone initiative successfully 
provided a model for Massachusetts to consider continuing in the 
future at the district or State level. 


identifying academic and non-academic needs, offer customized 

and multi-faceted interventions to at-risk students, and connect 
community services and organizations with students and families 
(e.g., health services, social workers). The State reported that LEAs 
have had the flexibility to implement plans tailored to their needs and 
that they make adjustments based on experience and lessons learned. 
Throughout the school year there were multiple opportunities for 
stakeholders to share best practices: district wraparound coordinators 
met three times; LEA and school leaders convened to share best 
practices and hear from experts; and ESE hosted content sessions on 
community partnership, effective family engagement, and creating 

a positive climate for students and staff. Each Wraparound Zone 
focused on sustainability in SY 2013-2014, and the State reported that 
each LEA has plans to maintain the personnel needed to support the 


Zone after the grant period. 


Having previously identified and vetted 23 Priority Partners to 
support Level 3, 4, and 5 schools, in Year 4 the State assessed the 
status of partners. Priority Partners are organizations that support 
district and school turnaround interventions and have a proven track 
record of results and have demonstrated effectiveness in improving 
student achievement. The State annually assesses Priority Partners 
through progress reports that demonstrate their effectiveness 

in accelerating school and LEA improvement efforts as well as 
conversations with schools and LEAs to determine their satisfaction 
with services. The State reported that all Level 4 districts are working 
with one or more Priority Partners and have expressed high levels of 
satisfaction. The Network of Priority Partners, launched in December 
2011, continued to convene throughout SY 2013-2014 as a venue for 
collaborating, coordinating, and aligning services across partners. In 
particular, Priority Partners worked to identify critical attributes of 


partnership and created guidance for partners working with districts 
and schools. Finally, the State continued to track the progress of seven 
Investment Fund grants awarded to Priority Partners to implement 
initiatives in Level 3, 4, and 5 schools and LEAs. In addition to 
vetting Priority Partners, the State identified and pre-approved 

eight turnaround providers to manage low-performing schools or 
implement intervention models to meet the needs of students in low- 
performing schools. 


Massachusetts continued in Year 4 to work with teacher and 

leader pipeline programs to recruit, train, and deploy teacher and 
leader teams into Level 3 and 4 schools. The State reported that 

the work of these teams has resulted in positive culture shifts and 
improved student outcomes. During Years 3 and 4 of the grant 
period, six candidates completed the Turnaround Leaders program 
for experienced leaders, were placed in Level 4 schools, and received 
onsite coaching supports. In the same timeframe, 10 aspiring leaders 
completed the Turnaround Leaders program and six were hired for 
leadership positions in low-performing schools beginning in 

SY 2014-2015. With 16 new leaders at low-performing schools, the 
State is short of its goal of 65 new leaders by the end of the grant 
period. Through the aspiring Turnaround Teacher Team initiative 56 
teachers were placed in two LEAs in SY 2013-2014, and an additional 
35 are planned for SY 2014-2015. The Turnaround Teacher Team 
initiative provided supports to teams of 6 to 15 teacher leaders in six 
schools in two LEAs. In one LEA the teams focused on using data to 
identify professional practice and team goals, and in the other LEA 
the teams focused on the needs of a Level 4 elementary school. 


Successes and challenges 


Massachusetts’ work to support the lowest-achieving schools 
continued to be strong in Year 4. The State’s approach to identifying 
and planning interventions for schools and LEAs continues to center 
on the core conditions for teachers, leaders, and students that impact 
success. The student outcomes data in Level 4 schools described above 
indicates the success of interventions and approaches to teaching and 
leading. Since the start of the grant period, 18 of 34 schools identified 
as Level 4 schools have exited Level 4 status. Evaluations planned 

for Year 5 may provide additional detail on the success of the State’s 
efforts. 


Though the State fell short of its goals for the number of Turnaround 
Leaders, it plans to continue and expand the program in SY 2014- 
2015. In addition, the State has sustainability plans in place for some 
turnaround supports, including the Wraparound Zones and District 
Governance Support Program. However, given that some Level 4 
schools are persisting in this status, it will be critical for the State to 
consider ways to continue school- and LEA-specific supports. 
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Emphasis on Science, Technology, 


Engineering, and Mathematics (STEM) 


Race to the Top States are committed to providing a high-quality plan with a rigorous course of study 
in STEM. In doing so, each State must cooperate with STEM-capable community partners in order to 
prepare and assist teachers in integrating STEM content across grades and disciplines, in promoting 
effective and relevant instruction, and in offering applied learning opportunities for students. A focus 
on STEM furthers the goal of preparing more students for an advanced study in sciences, technology, 
engineering, and mathematics, including among underrepresented groups such as female students. 


State’s STEM initiatives 


In SY 2013-2014 the State continued to provide technical assistance 
to six LEAs for planning and implementing STEM ECHS programs. 
Select middle and high schools partnered with local institutes of 
higher education to deliver a STEM-focused course of study for a 
group of students. Though each program varies in structure, they 

are all designed to target low-income, minority, and first-generation 
college-goers for postsecondary education and motivate them to 
explore STEM careers by providing flexibility to earn college credits 
while still in high school. School leaders engaged in networking 
meetings to share ideas and lessons learned, and in December 2013 
they participated in a joint convening with representatives from other 
early college and dual enrollment programs in the State. 


In SY 2013-2014, the State reported that 70 new candidates enrolled 
in the UTeach program at UMass-Lowell, bringing total participation 


in the program to 212 candidates. Though participation is increasing, 
the State is not on track to meet its goal of producing 250 STEM 
teachers through the UTeach program by the end of the grant period. 
Going forward, the State reported that UMass-Lowell has plans to 
sustain the program through fundraising after the grant period. 


Successes and challenges 


Together with other college- and career-ready initiatives for students 
and teachers, such as MassCore and pre-Advanced Placement training, 
the ECHS and UTeach programs demonstrate the State’s commitment 
to the STEM subject areas. The State continuously improved technical 
assistance for ECHS sites based on early lessons learned, and provided 
opportunities for practitioners to share best practices. However, while 
the UTeach program has increased participation, few candidates have 
completed the course. As such, the State fell short of its goal of 250 
STEM teachers by the end of the grant period. 


Looking Ahead 


Most Race to the Top States developed plans to continue their comprehensive reform efforts for an 
additional year (through the no-cost extension) and are developing plans to sustain many of their projects 


beyond the grant period. 


Massachusetts will continue multiple critical work streams into 

SY 2014-2015, the no-cost extension year after the initial four-year 
grant period. ESE will continue to support 46 of 233 Race to the 
Top participating LEAs that chose to develop Year 5 plans. These 
LEAs will leverage grant funds to support local efforts in areas such 

as curriculum work, technology investments, teacher trainings, and 
development of district-determined measures. The State plans to 
maintain personnel to manage the grant within ESE through 

SY 2014-2015. Across project areas and consistent with previous years, 
the State’s work in Year 5 is intended to continue to provide tools and 
resources that are available to LEAs well beyond the grant period. 


In Year 5, the State plans to continue supporting the standards 
transition with ongoing curriculum unit development and refinement 
and training on use of curriculum units and maps. In addition, ESE 


intends to continue expanding its video library of Massachusetts 
educators demonstrating classroom best practice in teaching the 
new standards. ‘The State also plans to enhance content currently in 
the Edwin Teaching and Learning system, including tagging items 
for inclusion in the interim and formative assessment bank, vetting 
additional locally-developed items, and providing guidance on how 
best to use assessment items. Going forward, the State reported that 
UMass-Lowell has plans to sustain the program through fundraising 
after the grand period. 


In SY 2014-2015, ESE intends for every participating LEA to comply 
with the interoperability framework, which will enable educators 

to use the Edwin Teaching and Learning system’s assessment and 
instructional resources. This milestone will be particularly critical 
given the challenges during the grant period. The State will continue 
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to provide onsite technical assistance to LEAs implementing the 
interoperability framework and plans to be able to collect student 
discipline data through the interoperability framework from all 

LEAs. Edwin Teaching and Learning will be available to LEAs at cost 
after SY 2014-2015. The State plans to implement a new approach 

to providing educators with data-driven instruction professional 
development. ESE intends to work with vendors to develop system- 
embedded modules for educators using data available in existing State 
data systems, like Edwin Analytics and Edwin Teaching and Learning. 
In addition, the State is developing plans to support Level 3 schools 
in Level 4 districts in spring 2015 with job-embedded coaching and 
professional development on using data to support instruction. As 
educator access to data increases in SY 2014-2015, these supports will 
be critical to making that data actionable in classrooms. 


Participating LEAs will be in their third year of educator evaluation 
system implementation in SY 2014-2015 and all other LEAs will be 
in their second year. ESE plans to continue providing supports to 
LEAs through guidance, trainings, and data collection and analysis. 
In addition, ESE staff intends to continue analyzing feedback from 
the field and evaluation data to make adjustments to implementation 
and plan for future supports. In Year 5, Massachusetts will pilot 


and implement a principal performance assessment and a teacher 
leader performance assessment, making progress on its goals related 
to improving the State’s licensure system. Also in Year 5, the State 
plans to provide extensive Sheltered English Immersion professional 
development to educators with English learner students and will 
continue the UTeach program at UMass-Lowell. 


Schools and districts identified as low-performing will continue to 
work with vetted turnaround partners, provide in-person professional 
development programs, and draw upon teacher and leader turnaround 
teams during Year 5. The State will build upon successes in 
Wraparound Zones by hosting cross-district seminars on approaches 
to wraparound services and other interventions. Further, the State 

will refine its protocols and management tools to identify and support 
Level 5 schools. 


‘The State and State partners have determined sustainability plans for 
some projects across the plan to ensure they continue after the four- 
year grant period. For example, LEAs with STEM ECHS programs 
are developing sustainability plans for SY 2014-2015 and UMass- 
Lowell has a fundraising plan to sustain their UTeach program. In 
addition, the State integrated sustainability into its approach to the 
range of supports implemented in turnaround schools and districts. 


Budget 


For the State’s expenditures through June 30, 2014, please see the APR Data Display at http://www.rtt-apr.us. 


For State budget information, see http://www2.ed.gov/programs/racetothetop/state-scope-of-work/index.html. 


For the State’s fiscal accountability and oversight report, see http://www2.ed.gov/programs/racetothetop/performance-fiscal-accountability.html. 
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Glossary 


Alternative routes to certification: Pathways to certification that 

are authorized under the State’s laws or regulations that allow the 
establishment and operation of teacher and administrator preparation 
programs in the State, and that have the following characteristics (in 
addition to standard features such as demonstration of subject-matter 
mastery, and high-quality instruction in pedagogy and in addressing 
the needs of all students in the classroom including English learners 
and students with disabilities): (1) can be provided by various types 
of qualified providers, including both institutions of higher education 
(IHEs) and other providers operating independently IHEs; (2) are 
selective in accepting candidates; (3) provide supervised, school-based 
experiences and ongoing support such as effective mentoring and 
coaching; (4) significantly limit the amount of coursework required or 
have options to test out of courses; and (5) upon completion, award 
the same level of certification that traditional preparation programs 
award upon completion. 


Amendment requests: In the event that adjustments are needed to 

a State’s approved Race to the Top plan, the grantee must submit 

an amendment request to the Department for consideration. Such 
requests may be prompted by an updated assessment of needs in that 
area, revised cost estimates, lessons learned from prior implementation 
efforts, or other circumstances. Grantees may propose revisions to 
goals, activities, timelines, budget, or annual targets, provided that 

the following conditions are met: the revisions do not result in the 
grantee’s failure to comply with the terms and conditions of this award 
and the program's statutory and regulatory provisions; the revisions do 
not change the overall scope and objectives of the approved proposal; 
and the Department and the grantee mutually agree in writing to 

the revisions. The Department has sole discretion to determine 
whether to approve the revisions or modifications. If approved by the 
Department, a letter with a description of the amendment and any 
relevant conditions will be sent notifying the grantee of approval. (For 
additional information, please see http://www2.ed.gov/programs/ 
racetothetop/amendments/index.html.) 


America COMPETES Act elements: The twelve indicators specified in 
section 6401(e)(2)(D) of the America COMPETES Acct are: 

(1) a unique statewide student identifier that does not permit a student 
to be individually identified by users of the system; (2) student-level 
enrollment, demographic, and program participation information; 

(3) student-level information about the points at which students 

exit, transfer in, transfer out, drop out, or complete P—16 education 
programs; (4) the capacity to communicate with higher education data 
systems; (5) a State data audit system assessing data quality, validity, 
and reliability; (6) yearly test records of individual students with respect 
to assessments under section 1111(b) of the Elementary and Secondary 
Education Act (ESEA) (20 U.S.C. 6311(b)); (7) information on 
students not tested by grade and subject; (8) a teacher identifier 

system with the ability to match teachers to students; (9) student-level 
transcript information, including information on courses completed 
and grades earned; (10) student-level college-readiness test scores; 


(11) information regarding the extent to which students transition 
successfully from secondary school to postsecondary education, 
including whether students enroll in remedial coursework; and 
(12) other information determined necessary to address alignment 
and adequate preparation for success in postsecondary education. 


American Recovery and Reinvestment Act of 2009 (ARRA): On 
February 17, 2009, President Obama signed into law the ARRA, 
historic legislation designed to stimulate the economy, support job 
creation, and invest in critical sectors, including education. The 
Department of Education received a $97.4 billion appropriation. 


Annual Performance Report (APR): Report submitted by each grantee 
with outcomes to date, performance against the measures established 
in its application, and other relevant data. The Department uses data 
included in the APRs to provide Congress and the public with detailed 
information regarding each State’s progress on meeting the goals 
outlined in its application. The annual State APRs are found at 
www.rtt-apr.us. 


College- and career-ready standards: State-developed standards that 
build toward college and career readiness by the time students graduate 


from high school. 


Common Core State Standards (CCSS): Kindergarten through 
twelfth grade (K-12) English language arts and mathematics standards 
developed in collaboration with a variety of stakeholders including 
governors, chief State school officers, content experts, teachers, school 
administrators, and parents. (For additional information, please see 
http://www.corestandards.org/). 


‘The education reform areas for Race to the Top: (1) Standards and 
Assessments: Adopting rigorous college- and career-ready standards 
and assessments that prepare students for success in college and career; 
(2) Data Systems to Support Instruction: Building data systems that 
measure student success and support educators and decision-makers in 
their efforts to improve instruction and increase student achievement; 
(3) Great Teachers and Great Leaders: Recruiting, developing, retaining, 
and rewarding effective teachers and principals; and (4) Turning 
Around the Lowest-Achieving Schools: Supporting local educational 
agencies’ (LEAs’) implementation of far-reaching reforms to turn 
around lowest-achieving schools by implementing school intervention 
models. 


Effective teacher: A teacher whose students achieve acceptable rates 
(e.g., at least one grade level in an academic year) of student growth (as 
defined in the Race to the Top requirements). States, LEAs, or schools 
must include multiple measures, provided that teacher effectiveness 

is evaluated, in significant part, by student growth (as defined in the 
Race to the Top requirements). Supplemental measures may include, 
for example, multiple observation-based assessments of teacher 
performance. 
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High-minority school: A school designation defined by the State in 
a manner consistent with its Teacher Equity Plan. The State should 
provide, in its Race to the Top application, the definition used. 


High-poverty school: Consistent with section 1111(h)(1)(C) (viii) 
of the ESEA, a school in the highest quartile of schools in the State 
with respect to poverty level, using a measure of poverty determined 
by the State. 


Highly effective teacher: A teacher whose students achieve high rates 
(e.g., one and one-half grade levels in an academic year) of student 
growth (as defined in the Race to the Top requirements). States, LEAs, 
or schools must include multiple measures, provided that teacher 
effectiveness is evaluated, in significant part, by student growth (as 
defined in the Race to the Top requirements). Supplemental measures 
may include, for example, multiple observation-based assessments 

of teacher performance or evidence of leadership roles (which may 
include mentoring or leading professional learning communities) that 
increase the effectiveness of other teachers in the school or LEA. 


Instructional improvement systems (IIS): Technology-based 

tools and other strategies that provide teachers, principals, and 
administrators with meaningful support and actionable data 

to systemically manage continuous instructional improvement, 
including such activities as instructional planning; gathering 
information (e.g, through formative assessments (as defined in the 
Race to the Top requirements), interim assessments (as defined in the 
Race to the Top requirements), summative assessments, and looking at 
student work and other student data); analyzing information with the 
support of rapid-time (as defined in the Race to the Top requirements) 
reporting; using this information to inform decisions on appropriate 
next instructional steps; and evaluating the effectiveness of the 

actions taken. Such systems promote collaborative problem-solving 
and action planning; they may also integrate instructional data 

with student-level data such as attendance, discipline, grades, credit 
accumulation, and student survey results to provide early warning 
indicators of a student's risk of educational failure. 


Invitational priorities: Areas of focus that the Department invited 
States to address in their Race to the Top applications. Applicants 
did not earn extra points for addressing these focus areas, but many 
grantees chose to create and fund activities to advance reforms in 
these areas. 


Involved LEAs: LEAs that choose to work with the State to implement 
those specific portions of the State’s plan that necessitate full or nearly- 
full statewide implementation, such as transitioning to a common set 
of K-12 standards (as defined in the Race to the Top requirements). 
Involved LEAs do not receive a share of the 50 percent of a State’s 
grant award that it must subgrant to LEAs in accordance with section 
14006(c) of the ARRA, but States may provide other funding to 
involved LEAs under the State’s Race to the Top grant in a manner that 
is consistent with the State's application. 


No-Cost Extension (Year 5): A no-cost extension provides grantees 
with additional time to spend their grants (until September 2015) to 
accomplish the reform goals, deliverables and commitments in its Race 
to the Top application and approved Scope of Work. Grantees made 
no-cost extension amendment requests to extend work beyond the final 
project year, consistent with the Amendment Principles (http://www2. 
ed.gov/programs/racetothetop/grant-amendment-submission-process- 
oct-4-2011.pdf) as well as the additional elements outlined in the 
Department Review section of the Amendment Requests with No Cost 
Extension Guidance and Principles document (http://www2.ed.gov/ 
programs/racetothetop/no-cost-extenstion-submission-process.pdf). 


Participating LEAs: LEAs that choose to work with the State to 
implement all or significant portions of the State’s Race to the Top plan, 
as specified in each LEA’s agreement with the State. Each participating 
LEA that receives funding under Title I, Part A will receive a share of 
the 50 percent of a State’s grant award that the State must subgrant to 
LEAs, based on the LEA’s relative share of Title I, Part A allocations 

in the most recent year at the time of the award, in accordance with 
section 14006(c) of the ARRA. Any participating LEA that does not 
receive funding under Title I, Part A (as well as one that does) may 
receive funding from the State’s other 50 percent of the grant award, in 
accordance with the State’s plan. 


The Partnership for Assessment of Readiness for College and 
Careers (PARCC): One of two consortia of States awarded grants under 
the Race to the Top Assessment program to develop next-generation 
assessment systems that are aligned to common K-12 English 

language and mathematics standards and that will accurately measure 
student progress toward college and career readiness. (For additional 
information, please see http://www.parcconline.org/.) 


Persistently lowest-achieving schools: As determined by the 

State, (1) any Title I school in improvement, corrective action, or 
restructuring that (a) is among the lowest-achieving five percent of 
Title I schools in improvement, corrective action, or restructuring or 
the lowest-achieving five Title I schools in improvement, corrective 
action, or restructuring in the State, whichever number of schools is 
greater; or (b) is a high school that has had a graduation rate as defined 
in 34 CFR 200.19(b) that is less than 60 percent over a number 

of years; and (2) any secondary school that is eligible for, but does 

not receive, Title I funds that (a) is among the lowest-achieving five 
percent of secondary schools or the lowest-achieving five secondary 
schools in the State that are eligible for, but do not receive, Title I 
funds, whichever number of schools is greater; or (b) is a high school 
that has had a graduation rate as defined in 34 CFR 200.19(b) that 

is less than 60 percent over a number of years. To identify the lowest- 
achieving schools, a State must take into account both (1) the academic 
achievement of the “all students” group in a school in terms of 
proficiency on the State’s assessments under section 1111(b)(3) of the 
ESEA in reading/language arts and mathematics combined; and (2) the 
school’s lack of progress on those assessments over a number of years in 
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the “all students” group. (For additional information, please see 
http://www2.ed.gov/programs/sif/index.html.) 


Qualifying evaluation systems: Educator evaluation systems that 
meet the following criteria: rigorous, transparent, and fair evaluation 
systems for teachers and principals that: (1) differentiate effectiveness 
using multiple rating categories that take into account data on student 
growth as a significant factor, and (2) are designed and developed with 
teacher and principal involvement. 


Reform Support Network (RSN): In partnership with the 
Implementation and Support Unit (ISU), the RSN offers collective and 
individualized technical assistance and resources to grantees of the Race 
to the Top education reform initiative. The RSN’s purpose is to support 
the Race to the Top grantees as they implement reforms in education 
policy and practice, learn from each other and build their capacity to 
sustain these reforms. 


‘The School Improvement Grants (SIG) program is authorized under 
section 1003(g) of Title I of the ESEA. Funds are awarded to States 
to help them turn around persistently lowest-achieving schools. (For 
additional information, please see http://www2.ed.gov/programs/sif/ 
index.html.) 


School intervention models: A State’s Race to the Top plan describes 
how it will support its LEAs in turning around the lowest-achieving 
schools by implementing one of the four school intervention models: 


¢ Turnaround model: Replace the principal and rehire no more 
than 50 percent of the staff and grant the principal sufficient 
operational flexibility (including in staffing, calendars/time and 
budgeting) to fully implement a comprehensive approach to 
substantially improve student outcomes. 


e Restart model: Convert a school or close and reopen it under a 
charter school operator, a charter management organization, or 
an education management organization that has been selected 
through a rigorous review process. 


® School closure: Close a school and enroll the students who 
attended that school in other schools in the district that are 
higher achieving. 


¢ Transformation model: Implement each of the following 
strategies: (1) replace the principal and take steps to 
increase teacher and school leader effectiveness, (2) institute 
comprehensive instructional reforms, (3) increase learning 
time and create community-oriented schools, and (4) provide 
operational flexibility and sustained support. 


Single sign-on: A user authentication process that permits a user to 
enter one name and password in order to access multiple applications. 


The SMARTER Balanced Assessment Consortium (Smarter 
Balanced): One of two consortia of States awarded grants under the 
Race to the Top Assessment program to develop next-generation 
assessment systems that are aligned to common K-12 English 

language and mathematics standards and that will accurately measure 
student progress toward college- and career-readiness. (For additional 
information, please see http://www.k12.wa.us/SMARTER/default.aspx.) 


‘The State Scope of Work: A detailed document for the State’s projects 
that reflects the grantee’s approved Race to the Top application. The 
State Scope of Work includes items such as the State’s specific goals, 
activities, timelines, budgets, key personnel, and annual targets for key 
performance measures. (For additional information, please see http:// 
www2.ed.gov/programs/racetothetop/state-scope-of-work/index.html.) 
Additionally, all participating LEAs are required to submit Scope of 
Work documents, consistent with State requirements, to the State for 
its review and approval. 


Statewide longitudinal data systems (SLDS): Data systems that 
enhance the ability of States to efficiently and accurately manage, 
analyze, and use education data, including individual student 
records. The SLDS help States, districts, schools, educators, and other 
stakeholders to make data-informed decisions to improve student 
learning and outcomes, as well as to facilitate research to increase 
student achievement and close achievement gaps. (For additional 
information, please see http://nces.ed.gov/Programs/SLDS/about_ 
SLDS.asp.) 


Student achievement: For the purposes of this report, student 
achievement (1) for tested grades and subjects is (a) a student's score on 
the State’s assessments under the ESEA; and, as appropriate, (b) other 
measures of student learning, such as those described in number 

(2) of this definition, provided they are rigorous and comparable across 
classrooms; and (2) for non-tested grades and subjects, alternative 
measures of student learning and performance such as student scores 
on pre-tests and end-of-course tests; student performance on English 
language proficiency assessments; and other measures of student 
achievement that are rigorous and comparable across classrooms. 


Student growth: The change in student achievement (as defined in the 
Race to the Top requirements) for an individual student between two 
or more points in time. A State may also include other measures that 
are rigorous and comparable across classrooms. 


Value-added models (VAMs): A specific type of growth model based 
on changes in test scores over time. VAMs are complex statistical 
models that generally attempt to take into account student or school 
background characteristics in order to isolate the amount of learning 
attributable to a specific teacher or school. Teachers or schools that 
produce more than typical or expected growth are said to “add value.” 
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